The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery

In 2012, Rivaie et al. introduced RMIL conjugate gradient (CG) method which is globally convergent under the exact line search. Later, Dai (2016) pointed out abnormality in the convergence result and thus, imposed certain restricted RMIL CG parameter as a remedy. In this paper, we suggest an efficient RMIL spectral CG method. The remarkable feature of this method is that, the convergence result is free from additional condition usually imposed on RMIL. Subsequently, the search direction is sufficiently descent independent of any line search technique. Thus, numerical experiments on some set of benchmark problems indicate that the method is promising and efficient. Furthermore, the efficiency of the proposed method is demonstrated on applications arising from arm robotic model and image restoration problems.


Introduction
In unconstrained optimization, large scale problem of the following form is usually considered, where the function f : R n ! R is smooth, meaning its gradient, g(x) = rf(x) is obtainable. Problems in the form of Eq (1) attracted much attention in optimization a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 community because of their wide variety of applications in different fields [1], including portfolio choice [2,3], m-tensor system [4], image restoration [5][6][7][8], signal recovery [9][10][11] and robotic motion [12][13][14] among others. The iterative methods that make use of gradient of f are usually preferable to solve these problems. Example of such methods is the Conjugate Gradient (CG) method, owing to its modest memory requirement and nice theoretical property. The method generates sequence {x k } by taking x 0 2 R n as an initial guess for the solution using the scheme The scalar α k > 0, called step-size or line-step is calculated so that it approximately meets the condition A practical technique used for obtaining the step-size is Wolfe rule [15]. The technique at k-th iteration is computed in such away that α k satisfies certain defined rules [16]. The commonly used technique referred as the standard line search technique, consisting the following inequalities Replacing Eq (5) with the formula gives the rule called strong Wolfe condition, where 0 < δ < σ < 1. Moreover, this step-size is calculated along the spectral search direction d k , given by in which, the spectral and CG updating parameters are represented by the scalars θ k and β k respectively. The parameter β k is crucial in constructing and determining choice of a CG method. Some earlier formulas for the parameters are proposed by Fletcher and Revees (FR) [17] and Polak, Ribière and Polyak (PRP) [18,19] with the following formulas respectively with y k−1 = g k−1 − g k and k.k represents ℓ 2 norm.
The RMIL method has been given considerable attention recently, but unfortunately, the convergence result of the RMIL is established based on exact line search, which is sometimes difficult to obtain. It has been pointed out by Dai (4) and (6). However, the Dai's condition may not always hold for general functions.
Another way to ensure that a CG method produces descent search directions is to scale the first term of the search direction, i.e. −g k , with a positive parameter, usually referred to as spectral parameter. Such modifications are known as spectral CG methods. For instance, Raydan [36] introduced a spectral CG method with good convergence properties. Also, spectral FR CG methods were suggested in [37,38]. The theoretical feature of these methods is that, regardless of which line search technique is employed, the direction d k satisfies Eq (9). The method in [37] reduces to classical FR parameter provided Eq (3) is satisfied, while that of [38] did not explicitly mention the line search rule used. Unfortunately, their numerical results are not promising even with inexact line minimization. The following references are available for more information on hybrid and spectral CG methods [27,[38][39][40][41][42][43][44][45][46][47].
Inspired by the contributions discussed so far, a spectral RMIL version is proposed in this paper. Interestingly, it is worth nothing that one of the major advantages of the proposed spectral RMIL method is that it possesses the nice restart feature of PRP method as well as strong convergence property of FR method. This is evident from the theoretical analysis presented in Section 3. In addition, the proposed spectral RMIL method does not require the restriction imposed on the RMIL method by Dai [34]. Combining these features together, one can see that the spectral RMIL method works better than the earlier restricted versions of RMIL method.
The key contributions of this paper are enumerated as follows: 1. An efficient CG parameter is suggested from spectral search direction.
2. The search direction is always descent without any line search consideration at every iterations.
3. Several large-scale benchmark unconstrained optimization problems from the literature are used to demonstrate the efficiency of the suggested scheme. 4. The algorithm is shown to converge globally under Lipschitz continuity condition.
5. Finally, the arm robotic model and image restoration problems are solved using the new spectral method.
The reminder of the paper is structured as follows. The new spectral CG method and its algorithm is presented next. In Section 3, the global convergence result of the new scheme via the standard Wolfe line search is analysed. The experimentation and comparison are discussed in Section 4 as well as applications of the new algorithm to solve arm motion control and image restoration problems in section 5 and 6 respectively and finally brief conclusion is made in the last section.

Algorithm and motivation
Consider the sequence {x k } generated by spectral CG method Eqs (2) and (7). Pre-multiplying Eq (7) by g T k and using Eq (8) together with Cauchy-Schwartz inequality gives For Eq (10) to satisfy Eq (9), implies that Applying Cauchy-Schwartz inequality on Eq (11), we have Thus, base on the above selection of the parameter θ k , the search direction d k is always descending independent of any line search rule, which means that the inequality Eq (9) is fulfilled. Therefore, motivated by this nice property, in this paper, we suggest the following spectral parameter Remark 1. The choice of parameter θ k in this form allows us to remove the computational burden and establish the global convergence of the proposed method without the condition, 0 � g T k g kÀ 1 � kg k k 2 imposed on the earlier versions of b RMIL k . In the following, the implementation procedure of the proposed method, NSRMIL method, is described.
Step 6: Update the next iterate from step 2.
To analyse Algorithm 1 convergence characteristics, the following presumption is useful 2. Denoting Γ as some neighborhood of δ, and function f is smooth with its gradient being Lipschitz continuous satisfying Under these assumptions, it is easy to see that for all x 2 δ, γ > 0.

Convergence analysis
We demonstrate the convergence of Algorithm 1 in this section. The Lemma that follows is taken from [25] and is an important part of the analysis. Lemma 3 Suppose that {x k } and {d k } are sequences generated by Algorithm 1, where the search direction d k is descent and α k fulfils Wolfe condition, then Theorem 4 If Assumption 2 holds and the sequence of iterates {x k } is produced by Algorithm 1, then Proof If Eq (17) does not hold, then there exists some constant m > 0 so that Claim: There exists a constant P > 0, such that the search direction specified by Eq (7) is bounded.
In order to establish the claim in Eq (19), which is crucial in showing the convergence of NSRMIL method, we need the following equations Eqs (8), (12), (13), (14), (15) and (18). Now taking the norm of the search direction given by Eq (7), we get Setting P ¼ Zg þ 2 LB Zm g; the result is established. The fourth and last line inequalities follow directly from Cauchy-Schwartz inequality and Assumption 2 respectively. Squaring both sides of Eq (11) gives Dividing through by kd k k 2 and summing gives which contradicts Eq (16). Thus, Eq (17) must hold.

Numerical results
Here, we give the numerical output of Algorithm 1 using some set of test functions considered by Andrei [24] and Jamil [48]. The problems are presented in Table 1 and compared with the following methods: • SPRP [49]: Algorithm 1 with b PRP k and y k ¼ d T kÀ 1 y kÀ 1 kg kÀ 1 k 2 .
• SCG [38]: Algorithm 1 with b FR k and y k ¼ À To analyse the performance of these coefficients, we consider forty nine (49) set of problems with different dimensions, and the codes are written in MATLAB 9.12 (R2022a), which is run on a personal computer with configuration; Intel(R) Core i7-1195G7 PC, RAM of 16 GB

PLOS ONE
The convergence of spectral RMIL conjugate gradient method with applications and 2.90 GHz of CPU. In running the code, we set parameters δ = 10 −4 and σ = 10 −3 in the strong Wolfe line search conditions for all the algorithms. The algorithms are set to stop when kg k k � 10 −6 or 2000 iterations is reached. To determine the proper value for the modulating parameter, η in the NSRMIL scheme, we tested the numerical behavior of the parameter for some selected values {0.0001, 0.01, 0.5, 1, 10, 20, 50, 100}, where the best result is obtained when η = 0.01. Furthermore, numerical results are compared using performance profile introduced by Dolan and Moré [50]. The detailed description of the numerical outcomes of the experiments are tabulated and available in S1 Table, and the results are presented graphically in Figs 1-3, where the P(τ) in the graphs denote the fraction of the test functions a method solved, and a method having high performance ratio of P(τ) is regarded the best. In addition, this ratio measures the performance of the algorithms based on number of iterations (NI), number of function evaluation (FE) and amount of time required to solve the problems (CPU). Similarly, the right hand part of the curves indicate robustness of a method. The interpretation of Figs 1-3 show that NSRMIL is efficient and preferable to other four methods. In Figs 1-3 we can see that, the NSRMIL method solves 88% of the test problem and win, i.e be the best in about 68% of the problems, followed by SRMIL+, RMIL+, SPRP and SCG methods respectively.

Application of NSRMIL on real-time 3DOF robotic model
In this section, we illustrate application of Algorithm 1 in solving the real-time motion control model of a three-joint planar robotic manipulator as investigated in [51]. The code and implementation of Algorithm 1 was performed using MATLAB R2022a 11 th Gen. Intel(R) Core i7-

PLOS ONE
The convergence of spectral RMIL conjugate gradient method with applications

PLOS ONE
The convergence of spectral RMIL conjugate gradient method with applications 1195G7 and run on a PC with RAM 16 GB that has CPU of 2.90 GHz. Briefly, the position level of discrete-time kinematics model equation is written as : ð24Þ The relation implies that h(�) is the kinematics mapping, which relates the orientation and position of any part of the robot. The length of the segments is denoted by of i th rod, and η i (for i = 1, 2, 3) and the vector b 2 R 3 of h(β) is the joint angles that show the end effector position. Let d k 2 R 2 indicate the desired travel vector at time interval t k 2 [0, t f ]. Therefore, in robotic model, we usually minimize the following nonlinear least square problem: where the vector δ k represents the actual path. So, the end-effector is controlled to monitor Lissajous curve [52]:

Application of NSRMIL for image recovery
The conjugate gradient (CG) method is widely known for solving large-scale smooth and nonsmooth convex optimization problems due to its efficiency and low memory requirement. A typical example of the smooth problem is the image restoration problem traced to the field of medical sciences, mathematical biology, and many more. This section tend to investigate the performance of Algorithm 1 in restoring images that have been corrupted by noise in the process of acquisition. One of the most common and frequently used noise restoration models is impulse noise (see; [53][54][55]). For the purpose of this study, we consider restoring the original 256 × 256 grey level images (x) of Canal and Building that have been corrupted by salt-andpepper impulse noise. To achieve this, we need to define the following terms. From the above statement, we denote the true image with M × N pixels by (x) and its index set define as W = {1, 2, . . ., M} × {1, 2, . . ., N}.
At the first stage, suppose we denote the detected set of indices of the noise pixels as K ¼ fði; jÞ 2 Wj � x ij 6 ¼ x ij ; x ij ¼ s min or s max g. Then, if we want to restore the noise pixels in the second stage, we have to minimize the following function: min GðuÞ;

PLOS ONE
The convergence of spectral RMIL conjugate gradient method with applications  Here, ϕ α denote the edge-preserving function defined as � a ðtÞ ¼ ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi t 2 þ a p and V ij = {(i, j − 1), (i, j + 1), (i − 1, j), (i + 1, j)} is the set of neighbours of the pixel at position (i, j). In the above function, � x denote an adaptive median filter to the observed corrupted noisy image ξ. In this part, the efficiency and relative accuracy of the proposed method is analysed based on peak signal-to-noise ratio (PSNR), relative error (RelErr), and CPU time (CPUT) using 30%, 50%, and 80% noise-degrees, respectively. The algorithms for this experiment are coded on MATLAB 2019a software installed on an Intel(R) Core(TM) i5-3210M PC with CPU@2.50 GHz, 4.00 GB RAM, and 64-bit.
The detailed description of the results are discussed in Tables 2-4, and the graphical representation of the restored images are presented in Figs 8 and 9. The results presented in the  Building image corrupted by 30, 50, and 80% salt-and-pepper noise: (a,b,c), the restored images using NSRMIL the least CPUT for a greater number of the noise degrees as seen in Table 2. This also applies to other metrics analysed in Tables 3 and 4. A close observation of the overall results show that the proposed method significantly outperformed the existing algorithms based on CPUT, RelErr, and PSNR. The NSRMIL method is able to de-correlates the grey noise and improved the correlation in signals with better accuracy.

Conclusion
In this paper, we have presented a simple spectral CG by incorporating update parameter in [33]. The parameter is obtained by modifying the spectral CG direction, where the sufficient descent condition and the global convergence hold independent of any line search rule. The theoretical analysis is not based on assumptions of earlier b RMIL k versions given in [2,34,35] respectively. The promising numerical result was obtained by relaxing the computational burden associated with the aforementioned versions. Similarly, the numerical result of the proposed algorithm is efficient, when the modulating parameter η = 0.01 compared to some known existing methods. The robustness of the NSRMIL is also demonstrated in solving image restoration problem and arm robotic planar problem. Future work include exploring the proposed method in two-step algorithms as presented in [56][57][58].
Supporting information S1